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FOREWORD 



In order to reach the goals of the present study, we depended on a number of 
people for guidance, assistance, and support. Chief Nathaniel Hamilton, of the North 
Chicago High School, served as an instructor/ test administrator during the instruc- 
tional and test administrative phase of the work. Mr. Lester Harmon, Superintend- 
ent of North Chicago Schools, kindly granted Chief Hamilton the permission to work 
with us. 



Dr. Victor Fields, Office of Naval Research, provided field liaison and guid- 
ance during the initial phases of this research effort. Dr. Marshall Farr has con- 
tinued to encourage our efforts and has provided helpful suggestions. LT Martin 
Boone provided assitance in administrative matters. 

Chief J. W. Kelt, of the Recruit Training Command's Classification Section 
at Great Lakes, handled subject assignments and administrative details. Chief R. G. 
Reynolds, also of the Recruit Training Command, provided technical assistance and 
arranged for Chief Hamilton to join our research team. LT C. H. Regner, of the 
Recruit Training Command, acted as our Liaison Officer. CAPT Symans, Command- 
ing Officer, Recruit Training Command, gave our project support and encouragement. 

LCDR D. S. Betz, Commanding Officer, Propulsion Engineering School, Great 
Lakes, permitted us to use his school's facilities. CWO R. E. Howery, Chief J. L. 
DeLong, and Chief H. E. Wilton, of the Machinist Mate A School at Great Lakes, all 
provided much needed technical assistance during the course of this project. 

CWO J. T. Dubose, CWO D. Romanek, CWO J. McCool, and Chief W. C. Villo, 
of the Damage Control School at the Philadelphia Navy Yard, helped us to contruct 
the criterion tests to be used in the evaluation of the Fleet performance of our sub- 
jects. CDR May, Director of the Damage Control School, arranged for these mem- 
bers of his staff to consult with us. 

Arthur I. Siegel 

Brian A. Bergman 

APPLIED PSYCHOLOGICAL SERVICES, INC. 

June 1972 
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CHAPTER I 



INTRODUCTION AND PURPOSE 



Within the recent past, traditional verbal measures of assessment have come 
under both legal and psychometric scrutiny. The legal criticisms are interwoven 
with concepts related to discrimination and fair employment, while the psychometric 
criticisms have been largely, but not exclusively, involved with differential validity 
and "culture fairness. " The present program is based on a concept related to "cul- 
ture fairness" or "culture freeness" but may be more properly associated with a 
"culture loaded" description. By culture loaded, we mean performance prediction 
on the basis of instruments that have been loaded in the job culture of interest. It is 
contended that if a person can learn to perform a job sample, he can also learn to 
perform the total job. Accordingly, the demonstrated ability to learn selected job 
aspects is employed as a predictor of ability to learn to perform the total job. The 
job sample learning situations (mini tests) involve no written learning materials. 
Hence, they maximally simulate the onrthe-job training situation in which a foreman 
instructs a journeyman in job performance. 



Legal Aspects of Employment Discrimination 

Chief Justice Warren Burg«r (1970), in writing the majority decision concern- 
ing the Duke Power vs. Griggs case, indicated that employment policies, even if non- 
discriminatory in intent, which ", . . 'freeze' che status quo of prior discriminatory 
employment practices" (p. 5) cannot be maintained. He qualified this statement by 
indicating that a person need not be hired purely because he was once discriminated 
against, or because he is a minority group member. Congress just wishes ". . . the 
removal of artificial, arbitrary, and unnecessary barriers when the barriers" (p. 6) 

. . .promote discrimination against certain groups of persons. If an employment 
practice is unrelated to job success, it is illegal. Just because a selection test was 
professionally developed does not mean that it can be used for employment purposes. 
Congress has said "...that any tests used must measure the person for the job and 
not the person in the abstract" (p. 12). 

The above statement means that any employment tests used must be shown 
to be related to specific aspects of behaviors common to the job in question. If em- 
ployers use tests or other instruments that measure abilities not required of the job, 
they are breaking the law. Ruch (1971) maintained that most intelligence tests meas- 
ure nonrelevant aspects of jobs and are therefore illegal. He recommended the use 
of unique factor tests measuring specific job skills. Enneis (1969b) has arrived at 
a similar conclusion. Specifically, he favored relating selection standards to job re- 
quirements. 



The American Psychological Association’s Task Force on Employment Test 
ing of Minority Groups (1969) listed most of the common sources of discriminatory 
bias of which employers should be aware; 



1. inadequate recruiting in minority schools or neigh- 
borhoods 

2. unfair preliminary selection by the receptionist 

3. application blanks which are clinically interpreted 

4. weighted application blanks which may not be valid 
for the minority group 

5. interviews which are open to . . conscious and un- 
conscious perceptual bias" (p. 640). Most interviewers 
look for subsequent interview facts which support their 
first impression, and they neglect facts which go against 
their first impressions. 

6. promotion practices in which the supervisor's appraisal 
of the minority employee may be influenced by racial 
characteristics and not job performance 

7. test anxiety, including unfamiliarity of the minority ap- 
plicant (or employee) with testing or inadequate adher- 
ence to optimal testing conditions 

8. test content (especially verbal content) whichis mainly 
derived from middle class culture 

9. unfair interpretation of test scores 

10. test content which is unrelated to the job requirements. 



The task force did not recommend the removal of cultural differences (e.g. , 
culture free tests, analysis of covariance), but rather the identification from among 
the low scorers those who can overcome their lack of experience and who can profit 
from extra training. The responsibility for the extra training rests with the employer. 
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Problems in Establishing Transethnic Group Test Fairness 

Krug (1966) listed and criticized some of the methods used to eliminate or con- 
trol for test score differences so that minority group members seeking employment 
will not be discriminated against. The most primitive method for promoting equal 
employment opportunity is through the use of double standards (when the mean test 
score of the minority group is significantly lower than the mean test score of the major- 
ity group). In this instance, the employer uses a lower cutoff score for the minority 
applicants and avoids the exclusion of an "unfair" proportion of the minority group 
members from employment. Variations on this theme havebeen proposed by Thorndike 
(1971) and Darlington (1971). Thorndike maintained that . ..."If one acknowledges 
that differences in average test performance. . . exist between populations A and B, 
then a judgment on test-fairness must rest on the inferences that are made from the 
test rather than on a comparison of mean scores in the two populations" (p. 63). 
Thorndike advocated setting two different cutting scores for different groups in order 
to achieve optimal fairness. Darlington (1971), on the other hand, would add a cer- 
tain number of points to the lower (minority) groupand then apply the same cutting 
score. The double standard method is, of course, no substitute for differential pre- 
diction of minority and nonminority performance. 

"Culture free, " "culture fair, " and "culture equivalent" tests have also been 
proposed as methods for performing fair across groups assessment. The main as- 
sumption of proponents of "culture free" tests is that the test content has the same 
meaning for all cultures. Unfortunately, this assumption can seldomly be met. In 
addition, the content of such tests makes them irrelevant for application assessment 
purposes. 

Another kind of test with more modest claims attached to it is the "culture 
fair" test. The "culture fair" test is assume to contain a set of stimuli which are 
appropriate for at least two cultures. Krug suggests that "culture common" (p. 33) 
would be a better description of this kind of test than "culture fair. " 

"Culture equivalent tests" are different from "culture fair" tests because they 
are not predicated on common material, but rather on material which tests the same 
concept using stimuli appropriate to different cultures. 

Boehm (1971) recently examined 13 research studies which reported either 
differential validity or single-group validity. Differential validity exists when, for 
'two or more groups, significant differences exist between predictor-criterion corre- 
lations and for one or both groups these coefficients are significantly different from 
zero. In single-group validity, the difference between validity coefficients for the 
two groups is not statistically significant, and only one coefficient is significantly dif- 
ferent from zero. Boehm's analysis indicated that; 
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1. Only 60 of 160 (37.5%) of the correlation coefficients 
reported in the 13 studies reviewed were statistically 
significant for either the white or the black ethnic group. 

2. In five of 13 studies, the predictors were used as selec- 
tion measures with a resultant restriction in range. These 
predictors will usually have low validity, and using them 
excludes a high proportion of Negroes at the beginning and 
reduces the number of subjects for this group. 

3. Only seven of the correlations were associated with dif- 
ferential validity. Many of these plus the single group 
validities can be attributed to inadequate sample sizes. 

When the N of both Negroes and whites was above 100, 
there was no differential or single-group validity. 

4. Job knowledge and performance test criteria tended to 
yield higher validity coefficients than supervisory ratings. 

5. In 22 of 27 instances of overall validity (correlation co- 
efficients of both groups significant), supervisory ratings 
or rankings were not used as performance criteria, but 

in 19 of 33 single-group validities a rating was used; there- 
fore, single-group and both-group validity are associated 
with the type of criteria used. 

Boehm concluded that single-group validity is associated with small sample 
size and a supervisory rating criterion. Both-group validity, though, is dependent 
upon large sample size and the use of performance of job knowledge tests as criteria. 
She therefore recommended that employers use more objective measures of employee 
performance rather than supervisory ratings or rankings. Bennett (1969) has arrived 
at a similar conclusion. 

One study illustrating some of the problems inherent in using rating scales as 
performance criteria was performed by Flaugher, Campbell, and Pike (1969). Super- 
visory ratings were examined to determine if the ethnic group membership of the 
ratee and the rater influenced assessment. One-hundred sixty-eight Negro and 296 
white medical technicians employed in VA hospitals were all rated at least twice and 
given a separate job knowledge test. On the nine rating scales considered collective- 
ly, whites were rated higher (p < . 01). Whites also scored higher on the job knowledge 
test (p < . 01). Generally, Negro supervisors rated Negro incumbents higher than did 
white supervisors (one-half a standard deviation). Negro supervisors, though, did 
not rate white incumbents higher or lower than white supervisors. 
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Bartlett and O'Leary (1969) presented four cases illustrating how valid- 
ity coefficients can yield erroneous predictions when a single coefficient is used to 
predict performance for two ethnic groups. 

The first situation is that in which there are significant differences between 
test scores and criterion scores for both groups. In this situation, the overall 
validity coefficient can be increased if the differences are in the same direction for 
both groups. 

A second instance, also discussed by Guion (1966), Kirkpatrick, Ewen, 
Barrett, and Katzell (1967), and Linn and Werts (197 1), is that in which the test 
scores for the Caucasian group are higher, but the criterion scores are equal so 
that the performance of Caucasians is overpredicted and the performance of Negroes 
is underpredicted. 

The third case exists when there is a difference between the groups in per- 
formance, but no difference in the predictor scores. In this situation, the perform- 
ance of the high performance group will be underpredicted and the performance of 
thelow performance group will be overpredicted. 

Finally, when there is a difference in opposing directions between the pre- 
dictor and the criterion in both groups, the combined validity coefficient may be 
negative, yet differential prediction is positive for each group separately. Using 
an overall validity coefficient, in this situation, would result in the selection of the 
worst performers from each group. 

Bartlett and O'Leary also discussed four instances in which differential valid- 
ity can occur. Some of these overlap, to some extent, with their single coefficient 
cases. First, a test can be valid for one group and not for the other group, but the 
mean scores on the criterion and the predictor may be the same for both groups. If 
the combined validity coefficient were used (if it was significant), one might select 
better persons from the valid group and erroneously conclude that the performance 
of the nonvalid group was inferior. The solution to this problem is to seek other 
valid predictors for the nonvalid group. Another case is that in which thegroups 
differ in average performance, but exhibit no significant difference in their predic- 
tor score differences. The third case is that in which differences exist between 
the means for the predictors, but no differences exist in the criterion means, re- 
sulting in differential validity for the two groups. A final and important case is that 
in which both the predictor and the criterion means are higher for one group. In this 
case, combining groups would increase the validity, but the increased validity is due 
to racial (or sex, etc. ) differences, and use of this is illegal. 
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Bartlett and O'Leary concluded that, in addition to searching for valid pre- 
dictors of minority group performance, employers should initiate training proce- 
dures to enhance the minority group's chance of success, 

Einhorn and Bass (1971) demonstrated that prediction is not necessarily bet- 
ter in a group with a higher validity coefficient over a group with a lower validity co- 
efficient. This situation obtains when the high validity group has a larger standard 
error of estimate than the low validity group. 

Enneis (1969a) stressed the control of skewness to increase the fairness of 
prediction. When the variance of the scores of the minority group is less than the 
variance of the scores of the majority group and a favorable selection ratio exists 
(1 in 10), then more nonminorities will be hired. Also, if the score distribution of 
nonminorities is negatively skewed and the score distribution of minorities positive- 
ly skewed, then the majority group will be favored. 

Bennett (1969) and Enneis (1969c) both indicated that concurrent validity stud- 
ies do not give a conservative estimate of predictive validity. On the other hand, 
many psychologists feel that, because of the restricted score range in concurrent 
validity studies, the correlation estimate is an underestimate of the predictive valid- 
ity coefficient. 



Studies into Differential Validity 

Foley (1971) wished to investigate whether or not the Officer Qualification 
Test (OQT) used by the Navy was biased against Negroes. Foley posited that the 
OQT is unfair if the regression equation used to predict success for Caucasians un- 
derpredicts the performance of Negroes. The Negro sample was compared with a 
matched white sample and with an unselected white sample. The OQT predicted per- 
formance in school (grade point average) for Negroes (r = . 29) and for a matched 
Caucasion group (r = . 48). Foley demonstrated that use of theCaucasion regression 
equation for Negroes resulted in a slight overprediction of Negro performance in Of- 
ficer Candidate School. 

Plag and Goffman (1967) found that educational level was a better predictor 
of performance in the Air Force (four year effectiveness, semiannual marks, ad- 
vancement, disciplinary and commendatory actions, and adjustment ratings) than 
the Armed Forces Qualification Test (AFQT). Race was found to be an ineffective 
predictor of performance. 
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Baehr, Saunders, Froemel, and Furcon(197 1) conducted alarge scale dif- 
ferential validity study of policemen in the city of Chicago. Although there are sev- 
eral serious statistical errors in this study, some of the findings are worth noting. 
These investigators found that the best overall prediction was obtained for the black 
group, and that different tests (some overlapping) predicted performance across 
both racial groups. 

Lopez (1966) suggested that different standards be used for prediction of per- 
formance across subcultural groups. He found that Negro toll collectors scored 
lower on several tests, but performed equal to whites on the job. Moreover, differ- 
ent predictors were related to performance across racial groups. 

Mitchell and Albright (1968) conducted a large scale validational study in a 
large southern plant using the Wonderlic Intelligence Test and a Biographical Infor- 
mation Blank as predictors. The criteria used by Mitchell and Albright were super- 
visory ratings, rankings, and turnover. These investigators found that the Wonder- 
lic failed twice as many blacks (54 per cent) as whites (27 per cent) and that it was 
not a valid predictor for either racial group. Scores on the Biographical Information 
Blank, though, correlated . 30 with turnover. 

In a study performed at IBM (Wollowick, Greenwood, & McNamara, 1969), 
576 administrative personnel were given "in house" tests of vocabulary, nonverbal 
reasoning, and arithmetic. Salary and rankings were used as performance criteria. 
Black employees (N = 60) were matched with three separate samples of white em- 
ployees in three different ways; (a) job and demographic variables, (b) supervisory 
evaluations, and (c) adjusted salary. For the second (b)and third (c) matched sam- 
ples, whites tended to have significantly higher test scores than blacks, even though 
they were matched on performance and salary. Also, the white groups yielded high- 
er validity coefficients than the black group. 

Grant and Bray (1970), in a recent study, used a task proficiency after train- 
ing criterionbecause they considered task proficiency to be uninfluenced by super- 
visory bias, peer pressures, or motivation. These writers asked five telephone 
companies to hire 100 employees, one-half of whom were black, and one-half of 
whom did not meet recommended standards on a battery of written tests. Each em- 
ployee was sent through a training program consisting of seven levels. At each 
training level, an employee took a pre-test. If he passed the pre-test, he went on 
to the next level of training. If he failed the pre-test, he took training at that level, 
followed by a post-test. The highest training level passed was used as the best per- 
formance criterion. All the prediction instruments correlated significantly (mostly 
at p < .01) with the highest level passed in the training program. The minority and 
nonminority correlations were all virtually identical. The SCAT and the Abstract 
Reasoning Test, gave a multiple of R of . 49, and these were subsequently used for 
hiring purposes. #• 
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Ruda and Albright (1968) conducted a study which illustrates some of the 
problems inherent in using a single validity coefficient to predict across racial 
groups. These investigators found that the Wonderlic was weighted more heav- 
ily (first hurdle) in the prediction of turnover than a BIB (second hurdle). The 
Wonderlic correlated -.34 with turnover for whites and + . 10 for blacks. This 
situation results in the employment of the whites who are most likely to turn- 
over and essentially chance prediction for blacks. The weighted application blank 
score correlated much better with turnover than the Wonderlic, as evidenced by a 
. 24 correlation for blacks and a . 18 correlation for whites. 

Qunlitulivc Differences in Intellectual Functioning and Performance 

Rimland (1969) using an idea similar to Jensen's (1969) differentiated between 
abstract intelligence ("g") (the ability to manipulate symbols and events mentally) 
and practical intelligence (the ability to sustain or perform simple tasks which sim- 
ulate a job). Rimland posited that these intellectual types are inversely related to 
one another so that an individual who is high on one will not be high on the other. 
Most traditional tests of an abstract nature represent acceptable predictors of aca- 
demic success, while practical performance tests are better predictors of job per- 
formance. Rimland found that practical performance tests correlated . 19 to . 37 
with job performance in the Navy, while the highest correlation between the AFQT 
and job performance was . 22. 

McFann (1969) indicated, after reviewing previous research, that the dif- 
ferences between high and low aptitude men in Basic Combat Training was not as 
marked on motor skills and proficiency tests. On these latter tests, category four 
personnel usually met standard. In a project SPECTRUM study, men representing 
high, middle, and low aptitude groups were selected and individualized training in- 
stituted using videotape, a one-to-one student-teacher ratio, and specialized train- 
ing. In some tasks, low aptitude men reached standard, but took from twice to four 
times as long, and in other cases, they failed to master thematerial at all. McFann 
also found the high aptitude group to learn equally well with lecture or individualized 
training, while low aptitude groups learned well with individualized training, but not 
with lecture; therefore, aptitude interacts with method of instruction. 

Taylor, Montague, and Hauke (1970) were critical of the Army's lock-step 
training procedures. They indicated that this type of training makes it difficult to 
train high and low aptitude personnel together. The high aptitude students are held 
back because the training is not enough of a challenge, while the low aptitude stu- 
dents fail to learn because the material is too difficult. These researchers devel- 
oped a miniaturized training sequence utilizing a varietyof different procedures. 

The subjects used were 350 low aptitude recruits,, 190 middle aptitude recruits, 
and 180 high aptitude recruits. The best training approach for high aptitude re- 
cruits was one without structure. High aptitude recruits should be given the ob- 
jectives of training, allowed to choose their own study methods, and make their 
own decision as to when they are ready for testing. Middle aptitude subjects derive 
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the most benefit from the same kind of training given to high aptitude subjects ex- 
cept that they prefer the presence of a live monitor. The training method which is 
most beneficial to low aptitude subjects has; 

1. complete structure 

2. instruction presented in small steps 

3. a slow rate of presentation 

4. a high rate of repetition 

5. an elementary language level 

6. content presented in a functional context with 
provision for practice 

7. a live instructor 

8. constant prompting and feedback 

9. meaningful extrinsic motivators 

Moore, MacNaughton, and Osburn (1969) indicated that nonverbal tests are 
not necessarily the least biased against minority groups. These writers gave both 
verbal and spatial (nonverbal) tests to Negro and white oil refinery applicants. Both 
racial groups were matched on age and education. The spatial (nonverbal) test was 
found to fail more Negroes than traditional tests. This supports the idea that the 
nonverbal approach may not be the most nondiscriminatory. 

Farr, O'Leary, Pfeiffer, Goldstein, & Bartlett (1971) attempted todevelop 
learning measures of performance. These writers indicated that ". . . differential 
reinforcement of basic ability patterns could result in various' minority groups be- 
ing at different points in the learning curve. Thus, if the Negro's cultural back- 
ground reinforced a pattern of abilities which differed from that of the white sub- 
group, he would not be at the same point on the learning curve as his 'equally cap- 
able' white counterpart" (p. 116). This differential reinforcement puts some mem- 
bers of minority groups at a lower point on the learning curve than members of the 
majority group. Current tests, then, are poor predictors of ability to learn. A 
learning situation as a measure of ability should not depend on past learning. Farr 
et al. used miniature learning tasks, derived directly from the criterion, to pre- 
dict criterion performance. Forty-six white and 48 Negro college students were 
used as subjects. The learning tests consisted of; a paired associates task, a con- 
cept learning task, and a principle learning task. In addition, the Wonderlic, a vo- 
cabulary test, an addition test, and a digit span test were also administered. The 
criterion was measured performance in a programmed instructural text in statistics. 
The results demonstrated that whites exhibited more gain over trials than blacks in 
the concept learning and principles learning tasks. There was r.o difference in the 
performance of Negroes and whites over trials on the paired associates learning. 
With regard to the criterion, the best overall predictors were the Wonderlic and the 
Vocabulary tests. The principles learning task and the concept learning task also 
predicted some of the criterion variables. The paired associates task was unrelated 
to the criterion. 
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Performance Comparative Studies 



A number of studies have also comparatively examined the performance of 
different groups. 

Fox, Taylor, and Caylor (1969) used training tasks to compare low, middle, 
and high aptitude subjects on: (a) visual monitoring, (b) rifle assembly, (c) missile 
preparation, (d) phonetic alphabet learning, (e) map plotting, and (f) combat plotting. 
The low aptitude group needed 2-4 times more training time, 2-5 times more train- 
ing trials, and 2-6 times more prompting than the middle and high aptitude subjects. 
The middle aptitude group performed more like the high aptitude group than the low 
aptitude group. The authors urged that training programs be designed which account 
for these individual differences. 

Guinn, Tupes, and Alley (1970) examined training performance ingroups dif- 
fering in race, education, and area of the country for several occupational special- 
ties. Differences in training performance were found when the groups were divided 
on the basis of these variables, but differences were not found for all of the occu- 
pational specialties on the three variables of interest. 

Grunzke, Guinn, and Stauffer (1970) performed a followup study of 26, 915 
low aptitude (category IV) men accepted into the Air Force. After a comparison 
with normal enlistees, it was found that low aptitude men: 

1. were less likely to complete basic training 

2. had more unsuitable discharges 

3. were less likely to attain required skill levels. 

Van Matre and Harrigan (1970) compared the performance of 54 marginally 
qualified electronic technicians with 51 well-qualified electronic technicians who 
underwent training. Performance ratings were obtained in the Fleet on all 105 sub- 
jects after they were on the job for 24 months. The low aptitude group was rated 
similar to the normal group, with none lower than average. Generally, though, the 
higher aptitude group was rated as more capable in "trouble shooting" and "use of 
test equipment. " 

In another study. Van Matre and Steineman (1966) trained 26 low aptitude 
men in an electronics technician course in an abbreviated period of time. The men 
were only taught skills considered to be more immediately useful on the job. This 
experimental group was compared, in a six month followup, with 24 conventionally 
trained, non-low aptitude subjects. The results demonstrated that the performance 
of the low aptitude group was adequate and not significantly different from the con- 
ventional group. 




10 

c i ■ 



21 



Van Matre (1971) developed an instrument reading training course for low 
aptitude (category IV) personnel. Instrument reading is an ability required in many 
Fleet jobs in which category IV's take part. One-hundred and eighty-eight low ap- 
titude subjects were trained by a variety of methods; classroom lecture, on-the-job 
training, modified classroom, and independent self study with workbooks. Evalu- 
ations were in the form of written criterion tests taken before and after training. 

The results of this effort demonstrated that group IV men could be trained to the 
level of experienced non-group IV men. The most effective training method was 
self study with workbooks. 

Hooprich (1968) sought to determine the appropriateness of commisaryman 
training for category IV personnel. His conclusions, based on two successive stud- 
ies, were; 

1. 31 of35 category IV's successfully completed train- 
ing 

2. the grades of category IV's, though, were significantly 
lower than the grades of noncategory IV's 

3. thelow aptitude men needed to devote more outside 
time to study and they required more time from in- 
structors to meet criteria 

4. the differences between the category IV's and the high- 
er aptitude men were most evident on paper and pencil 
tests, and least evident on actual performance tests 

5. AFQT scores did not predict school performance. 

Standlee and Saylor (1969) performed a similar study with equipment oper- 
ators and obtained identical results. 

Rohwer, Ammon, Suzuki, and Levin (1971) worked with 288 elementary school 
children divided equally over kindergarten, first, and third grades, and also over 
middle to high socioeconomic white and low socioeconomic black groups. All subjects 
were given the Peabody Picture Vocabulary Test, the Raven Progressive Matrices, 
and a Paired-Associates Test. The results showed differences between whitesand 
blacks at all grade levels on the Peabody Picture Vocabulary Test and the Raven 
Progressive Matrices, thus supporting the conclusion that differences in school 
achievement maybe due to a learning deficiency. The whites, though, were signifi- 
cantly better than blacks only at the kindergarten level on the Paired-Associates 
test. These authors suggested that the other tests may require ". . . the mastery of 
sets of formal conventions (e. g. , numbers and categories) created by cultural con- 
sensus that may be more readily available to, or more valued by, one population 
than by another" (p. 13). 
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Motivational Co nsi derations 



One of the critiques leveled at Jensen's (1969) thesis that test score differ- 
ences between Negroes and Caucasions cannot be accounted for by the environment’ 
alone is that motivational differences between the races was not investigated. Jensen 
discussed motivation in his monograph, but not in as sophisticated a manner as he 
discussed genetics and environment. Several theories of motivation exist which may 
account for some of the variation in test scores between races that Jenscnattributes 
to genetic endowment. 

Rotter (1966) conceives the effect of reinforcement on behavior as dependent 
upon whether the person perceives a causal relation between his own behavior and the 
reward. If not, the reward is attributed to luck and to the control of others. Internal 
control exists when the subject thinks reinforcement is contingent upon his own be- 
havior, while external control exists when the subject thinks reinforcement is con- 
trolled by others or by chance events. In social learning theory, reinforcement in- 
creases the expectation that behavior is followed by a reinforcement. Failure of re- 
inforcement extinguishes this expectancy. Children, during development, will thus 
begin to distinguish causal from noncausal events, and these expectancies will con- 
trol choice behavior and performance. A person will perceive a reinforcement se- 
quence as not being chance controlled when the proportion of reinforcement is sig- 
nificantly different from 50:50 in a right-wrong situation. When the reinforcements 
are patterned and when variability is minimal in a task allowing great variability, 
the reinforcement is perceived as determined by others. 

It seems that internal-external control should be considered as antecedent 
factors in applicant or employee assessment. The only meaningful results would be 
obtained with subjects who are internally controlled. It is also easily seen that in- 
ternal-external control can be an important covariate that accounts for differences 
in intelligence test scores between deprived and nondeprived groups. 

In one study investigating the internal -external control concept (Scott & Phelan, 
1969), Rotter's Internal -External Control Scale was administered to threegroups of 
subjects. The subjects in all three groups were matched on age, socioeconomic sta- 
tus, and scholastic aptitudes. The results demonstrated that blacks and Mexican 
Americans demonstrated greater external control than whites. The authors con- 
cluded that the externally controlled subjects did not think that there was a relation- 
ship between individual effort and reward; therefore, they didn't work unless given 
external reinforcement (praise, money, etc.). 

Battle and Rotter (1963) used Rotter's Internal -External Control Scale to 
measure external control in several groups of Negroes and whites differing in socio- 
economic status. They found that lower class Negroes were higher on external con- 
trol than lower class whites, middle class Negroes, and middle class whites. Per- 
haps the perception of limited material opportunities and of powerful external forces 
produce an external control attitude. 
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Seeman (1963) and Seeman and Evans (1962) introduced the concept of aliena- 
tion, which they suggest to be measured by internal -external control. Alienation is 
a feeling of powerlessness or inability to control outcomes, and it is inversely re- 
lated to knowledge about a situation. In one study, using reform school subjects, 
Seeman (1963) sought to determine if poor learning was produced by powerlessness, 
or if powerlessness comes from poor learning. He found that alienation affected 
learning about parole information; he therefore concluded that expectancies govern 
attention and acquisition of knowledge. 

Rosenhan (1966) posited that lower class children are more alienated from the 
environment than middle class children in a middle class school system. Rosenhan 
then hypothesized that lower class children would be more responsive to praise in a 
binary choice game than middle class children, and that lower class performance 
would be more disrupted by disapproval than middle class performance. Rosenhan 
found an interaction between class and approval-disapproval. Over trials, the lower 
class approval group started by performing lower than the middle class groups, but 
finished higher than both middle class groups. The performanceof the lower class 
disapproval group, though, was disrupted in that they remained at a low level across 
trials. Also, performance of both middle class groups remained the same through- 
out the trials. From these results, Rosenhan concluded that the lower class child is 
unfamiliar with middle class institutions, and therefore more alienated and very re- 
sponsive to external social reinforcement. The middle class child, though, doesn't 
need external indices of performance. Continual disapproval, then, can have long 
term deletecictus effects for lower class children. 

Atkinson (1966) presented a somewhat more rigorous theory of motivation in- 
volving achievement motivation, incentive, and goal expectancy. Atkinson's theory 
is depicted in the following formula: 



Motivation = f (motive x expectancy x incentive) 



With nAch (motivation to approach a goal) held constant at 1. 00 and with expectancy 
and incentive equal to .5, then the probability of goal approachis . 25 (the highest 
possible). Atkinson defines incentive as goal attractiveness, and motive as the abil- 
ity to strive for satisfaction or to accomplish. "The strength of motivation to ap- 
proach decreases as probability of success increases from .50 to near certainty (P s = 

. 90), and it also decreases as P g decreases from . 50 to certainty of failure (P g =.10)" 
(p. 17). 

From the above formulation, it is easily seen that the young, deprived black 
child will rarely encounter a probability of success of .5 or greater. Because heper- 
ceives a certainty of failure, he then lacks the motivation to approach a goal, and 
therefore he does not perform as well in assessment situations as the nondeprived 
white child who perceives a higher probability of success. 
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Several recent studies were performed showing class and race differences 
in nAch. For instance, Rosen (1959) found nAch to be lower in Negroes from mid- 
dle and lower social classes than other ethnic and racial groups. Negroes in the 
upper social classes, though, were high in nAch. Rosen (1956), in another study, 
found that nAch increased as social class increased in white high school males. 
Mingione (1965) found among low socioeconomic groups that whites had higher nAch 
scores (p< . 001) on the TAT than Negroes. Finally, Shrivasta and Tiwari (1967) 
observed higher nAch in middle class children than in lower class children. 

Katz (1967) more or less integrated certain earlier theories into a coherent 
two-stage theory of development which has strong implications for assessment. Dur- 
ing the first stage (up to two years of age) of development, the child's verbal efforts 
are normally reinforced by parental approval. Selective approval, on the part of 
the parents, can develop strong habits of striving for proficiency in the child. Dur- 
ing stage two, the parental standards and values of achievement are internalized by 
the child. "The child's own implicit verbal responses acquire through repeated as- 
sociation with the overt responses of the parents the same power to guide and rein- 
force the child's own achievement behaviors. ... Internationalization doesn't take 
place until strong externally reinforced achieving habits have developed" (p. 5). 
Lower class children (including most blacks) are more dependent on others for soci- 
al reinforcement in academic situations. Lacking internalization, they will avoid 
achievement situations and concentrate on other situations regarded as more prom- 
ising. "Lower class Negro children tend to be externally oriented insituations that 
demand performance. That is, they are likely to be highly dependent on the imme- 
diate environment for the setting of standards and the dispensing of rewards" (p. 8). 
Achievement motives and dependency motives must therefore be accounted for in 
employee and applicant appraised programs. 

Crandall and his associates (Crandall, Preston, & Rabson, 1960; Crandall, 
Katkovsky, & Preston, 1962; Katkovsky, Crandall, & Good, 1967) have also at- 
tempted an integrated theory of motivation. They found that warm, praising, pro- 
tective, and supportive parental behaviors fostered child belief in interned control, 
while dominant, rejecting, and critical parental behaviors were negatively associ- 
ated with internal control. Also, children whose achievement efforts were rewarded 
as young children later come to value achievement activities as sources of satisfac- 
tion. 



Hess and Shipman (1965) presented a very interesting development formula- 
tion which goes further than the previously mentioned conceptualizations in explain- 
ing the differences between Negro and white test scores. These writers indicated 
that, cognitive growth is ", . . fostered in family control systems which offer and per- 
mit a wide range of alternatives of action and thought and that such growth is con- 
stricted by systems of control which offer predetermined solutions and few alterna- 
tives for consideration and choice" (p. 870). In the deprived family context, the 
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parent-child control system. . . " restricts the number and kind of alternatives for 
action and thought that are open to the child: such constriction precludes a tendency 
for the child to reflect, to consider, and choose among alternatives for speech and 
action. It develops modes for dealing with stimuli and with problems which are im- 
pulsive rather than reflective, which deal with the immediate rather than the future, 
and which are disconnected rather than sequential" (p. 870-871). Hess and Shipman 
concluded that the family shapes the modes of communication in the child which, in 
turn, shape his thought and problem solving style. 

Another motivational conception was presented by Coleand Bruner (1971). 
These writers dismissed the idea that one group is culturally superior to another. 
They indicated that what really exists is cultural differences. Persons who are 
classed as culturally deprived are not presented with situations in which they can 
demonstrate their skills. Instead, they are continually measured via the middle 
class culture. This culture is not coincident with their experience and is, accord- 
ingly, irrelevant to testing situations. Cole and Bruner conclude that ". . . cultural 
differences reside more in differences in the situations to which different cultural 
groups apply their skills than to differences in the skills possessed by the groups 
in question. ... cultural deprivation represents a special case of cultural difference 
that arises when an individual is faced with demands to perform in a manner incon- 
sistent with his past (cultural) experiences" (p. 874). 

In the final two studies reported in this section, the first (Friedrichs, Hertz, 
Moynahan, Simpson, Arnold, Christy, Cooper, & Stevenson, 1971) used five year • 
old middle and upper middle class children as subjects, while the second study 
(Stevenson, Williams, & Coleman, 1971) used lower class disadvantaged (mostly 
Negro) children as subjects. All subjects in both studies were given the same eight 
learning tasks. Some of the learning tasks were cognitive, whileothers were as- 
sociative, therefore providing a good test of Jensen's (1969) thesis. The resulting 
overall pattern of correlations for the learning tasks was very similar across groups. 
The authors concluded that ". . . there is little utility in positing differences between 
the two groups in the operation of associative and cognitive learning abilities" (p.183). 

In summation, these positions reveal with devastating clarity why Jensen 
(1969) seems to be incorrect in his ascription of heredity as partially causing Negro- 
white test score differences. This does not mean, though, that the cognitive styles 
of deprived and nondeprived persons are the same, as Hess and Shipman (1965), and 
Katz (1967) have so aptly pointed out. Clearly, in training program development, 
applicant appraisal, and employee development, these differences in cognitive style 
and motivation must be accounted for and taken into consideration so that the poten- 
tial of the human resources in our society can be maximized. Motivation and cog- 
nitive style variables should therefore be controlled in any studyin which racial test 
score differences are considered. If these factors are not assessed, regardless of 
how many other variables are included, one's conclusions are apt to be misleading 
or erroneous. 
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Discussion 



The literature reviewed suggests that for most testing situations, differential 
validity will exist across subcultures. Accordingly, differential testing methods and 
different regression equations are indicated for each subculture. 

One solution may rest in the "mini aptitude" test approachdescribed in sub- 
sequent sections of this report. Here, advantage is take of: (1) the concrete type 
thinking attributed by others to culturally deprived individuals, (2) the motivational 
aspects since the paper-and-pencil approach is avoided, and (3) the minimization of 
culturally loaded content. 

One problem is that not all minority group members have led a deprived ex- 
istence. How can we determine if a Negro or a Spanish American is deprived? Cer- 
tainly skin color and other physiognomic characteristics are associated with depri- 
vation, but they do not tell us with certainty that any one individual is disadvantaged. 

A solution is to use a well-constructed Biographical Information Blank (BIB) con- 
taining questions related to cultural exposure. The Negro child who owned 200 book% 
and who visited museums as a child is definitely less deprived than the Negro child 
who rarely saw a newspaper, much less a book. Many of the differential prediction 
studies may have "missed the mark, " because they haveused race as a moderator 
variable rather than cultural deprivation. The only reason some of these differenti- 
al prediction studies have been successful is because race is correlated with cultural 
deprivation. Naturally, more Negroes will be in the culturally deprived group, but 
some whites will be there, too. 

Kimble (1971) is one of the few investigators who have successfully devel- 
oped a cultural deprivation scale. Kimble's scale was based on three classesof 
variables: (1) amount, (2) variety, and (3) organization of stimulation. Thisscale 
was administered along with the School and College Abilities Test (SCAT) to 200 
students in remedial classes at a junior college. Kimble found that the subscales 
! of his cultural deprivation scale correlated between -.32 and -.44 with the SCAT. 

i 

Purpose of Present Work 

The primary purpose of the present work is to investigate, in the Navy con- 
text, assessment methods, techniques, and procedures which are free from the 
biases ascribed to more traditional testing approaches. The study does not focus 
; on a test or tests which possess equal predictive validity for both high and low apti- ; 

tude personnel. Rather, the assumption is made thatthe normal Navy testing vehi- ; 

cles (GCT + ARI + MECH + CLER) are adequate for persons possessing high apti- ■ 

tude as measured by these methods. The test results of persons who achievehigh j 
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scores on these instruments have not been affected to the extent that their progress 
in the military will be debilitated in any way. On the other hand, the results of per- 
sons who score poorly on these tests may be unduly affected by the factors discussed 
in earlier sections of this report. Accordingly, the present study attempts to devel- 
op predictive procedures which will identify low aptitude personnel (as measured by 
the usual Navy tests) who can perform adequately on the job. 

The underlying working hypothesis is that persons who exhibit the ability to 
learn sample aspects of a Navy job will be able to learn the total job, provided that 
they are given proper on-the-job training. A similar concept has been previously 
developed by Jensen (1969) and by Farr, O'Leary, Pfeiffer, Goldstein, and Bartlett 
(1971). 



The specific research steps include; 

1. development of a sample of miniaturized job learning 

, ,, situations (tests) for low aptitude personnel in the ma- 

chinist mate (MM) rating. These miniaturized job learn- 
ing situations are called training and evaluation situations 
in subsequent sections of this report 

2. administration of these tests to a sample of low aptitude 
black and white persons and assigning these persons to 
probable successful and probable unsuccessful groups 
on the basis of their test scores 

3. assignment of all persons sampled to Fleet jobs in the 
machinist mate rate 

4. followup to determine the degree of on-the-job success 
experienced by all persons in the sample.,- 

To date, steps 1, 2, and 3. have been completed. The methods, procedures, 
and results of these steps are reported in subsequent sections of this report. 

Followup in the Fleet (step 4) has not yet been accomplished. Followup stud- 
ies are planned after the tested individuals have been on their assigned jobs 6 months, 
12 months, and 24 months. 



CHAPTER II 



METHODS 



After discussion with persons occupying a number of desks concerned with 
Navy personnel and training methods and procedures, the machinist mate job was 
adopted as at least a logical starting place for a st®dy such as that which is here in- 
volved. The machinist mate rate in the Navy involves performance of tasks which 
are largely nonverbal in character. The thinking processes involved in these tasks 
are largely concrete (as opposed to abstract) in nature. Moreover, this rate is one 
in which there is not a large number of blacks. Nonetheless, it is a rate which 
should be attractive to most recruits since it is adequately high on the informal 
prestige scale for various Navy jobs and because it offers the potential for learning 
skills which can lead to post Navy employment. 

The normal entry into the machinist mate career field in the Navy is through 
the Navy "A" school for machinist mates. This school involves training in the funda- 
mental skills and knowledges required for performance at an entry level in this rate. 




Subj ec Is 

The subjects were Navy recuits who were identified after initial Navy testing 
by the Recruit Training Command at the Great Lakes Naval Training Center. Asa 
basic requirement for participation, a subject had to have "failed"* the entry tests 
for the Machinist Mate School. Ninety-nine recruits were so identified. Fifty of 
these recruits were white and 49 were black. Virtually all of the recruits were be- 
tween 19 and 20 years of age. 



Miniature Training and Evaluation Situations 

As a first step in the construction of the miniature training and evaluation 
situations, the machinist mate section of NAVPERS 18068A was consulted. Those 
practiced, behaviors required for advancement to level E-4 were extracted. Several 
of these behaviors were combined because they were of a similar nature. The next 
step involved a meeting with five Master Chief Machinist Mates and one warrant of- 
ficer at the Great Lakes Naval Training Center. During this meeting, a final list of 
behaviors, which were adequately representative of the most frequently performed 
or critical tasks of the journeyman level machinist mate, were agreed on. 



A recruit east exceed a combined C CT, A R l , and UECH score of 
156 to be eligible for entry to the Machinist Mate Class A 
School • 
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The behaviors so identified were: 



1. ability to identify and use hand tools common to job 

2. ability to perform maintenance and to read meters' 
and gauges accurately when under some degree of 
distraction, vigilance, or when attention sharing is 
involved 

3. ability to make simple repairs in pressure lines 

4. ability to perform simple troubleshooting and sys- 
tems analysis in pressure systems 

5. ability to operate equipment common to rate 

6. ability to assemble and disassemble common high 
failure frequency items 



These tasks formed the basis for the miniature training and evaluation situations. 
Several Master Chief Machinist Mates then served as technical consultants during 
the actual lesson construction phase of the project. This procedure resulted in 
the construction of six miniature training and evaluation lessons. These training 
and evaluation situations reflected samples of the most critical and/ or frequently 
performed behaviors of the journeyman machinist mate. 

Each training and evaluation situation contained two segments. The first 
segment, a training phase, usually involved a "show and tell "learning situationof 
15 to 30 minutes. All of these lessons were completely devoid of reading and writ- 
ing requirements.* The sections which follow describe each of the training and eval- 
uation situations in detail. 



Equipment Use and Nomenclature 

In the equipmentuse and nomenclature mini job sample situation, the objective 
was to determine if the recruits could learn the names and uses of all of the equipment 
and material involved in making-breaking a flange. It was assumed that if a sailor 
could learn the names and uses of the tools and materials involved in this situation, 
he would also be able to learn the names and uses of other equipment used on the job. 
The materials used in performing this repair task are:' (a) bolts, (b) nuts, (c) flanges. 



* Soae of the teats involved the ability to tell tlae and the ability to read 
nuabere. Ttiece,are consider ted to be preltierate requirements. 
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(d) drift pins, (e) gasket knife, (f) gasket material, (g) graphite grease, (h) ball 
pean hammer, and (i) box end wrenches. After a tape recorded introduction, the 
instructor demonstrated how to make and break a flange. During this demonstra- 
tion, the use and name of each piece of equipment was discussed by the instructor. 
For example, before using a particular tool, the instructor would holdup the tool 

and say "This is a . " He would then pass the tool among the students for 

their inspection. When the tool was returned to the instructor, he would demon- 
strate its use in the flange repair situation. Upon completion of the demonstration, 
a 25 question true-false test was administered to the recruits. For each item in 
this test, the instructor held up an object and ascribed a name or use to it. The 
recruits then indicated whether the name or use given by the instructor was true or 
false by encircling either the word "true" or the word "false" next to the item num- 
ber on their answer sheets. Each item was read twice with a 10 second interval be- 
tween items. 



Gasket Cutting and Meter Reading 

The. gasket cutting and meter reading training and evaluation situation was 
designed to investigate ability to learn a maintenance task and to perform when some 
degree of distraction or attention sharing is involved. This situation was also de- 
signed to sample the vigilance situation in which the machinist mate, on the job, must 
monitor the states of various equipment systems while he performs other tasks. Af- 
ter a tape recorded introduction, the subjects were taught, through demonstration, 
how to make a gasket using a flange, a ball pean hammer, asbestos gasket mate- 
rial, and some bolts. In this demonstration, the gasket material was placed over 
the flange so that the faceof the flange was entirely covered. Then, usingthe round 
end of the ball pean hammer, one bolt hole was tapped out. Only light taps were sug- 
gested, because heavy hammering on the flange would eventually damage it. Next, a 
bolt was placed in the bolt hole. The next step was to tap out a bolt hole, exactly op- 
posite the first one. This was done so that the gasket would not shift over the flange. 
The remaining bolt holes were then tapped out in any order. Theinner circumfer- 
ence was then tapped out, again using the round end of the ball pean hammer. The 
flat end of the ball pean hammer was then used to tap out the gasket material remain- 
ing over the outside edge of the flange. Finally, the tapped out pieces of gasket ma- 
terial were removed from the flange and the gasket. 

Upon completion of the lesson, the subjects were given a 10 minute gasket 
making practice session. During the practice sessions, the instructor circulated 
among the recuits and assisted them as required. 
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Next, the subjects were taught how to read a pressure meter and: (1) how to 
log the time at which the pressure deviated from the normal, and (2) whether the 
pressure should be adjusted to go up or down (relative to a given nominal value). A 
large clock with a sweep hand was placed in front of the testing room. The recruits 
read the time from this clock. 

The tests for both gasket cutting and meter reading were administered togeth- 
er. That is, for a tenrminute period, the subject had to observe and record from a 
meter while he constructed a gasket. The meter which each student read was placed 
at his individual work station. Each meter was individually driven so that there was 
no possibility for a subject who noticed a system out-of-tolerance condition at his 
station to cue a recruit at another station of an out-of-tolerance condition at the sec- 
ond station. However, the signal presentations to all stations were equated for num- 
ber, direction, and magnitude of deviation. This combination performance testing 
introduced the required attention sharing component into the criterion situation. 

The gasket making was scored through a checklist which was completed by 
the instructor as the recruit performed the task. The scoring checklist included 
'items on adherence to correct procedures, care and use of tools, adherence to safety 
precautions, and adequacy of the final gasket. The meter reading aspect was scored 
on the basis of: (1) number of out-of-tolerance conditions correctly noted, (2) correct 
ness of indication of needed pressure adjustment (up or down) to restore system to 
nominal, and (3) precision of log entry for time of deviation. 

The scoring checklist for the gasket cutting is included in Appendix A to this 
report. Figure 2-1 shows the black instructor demonstrating the gasket making pro- 
cedure. Figure 2-2 shows one of the recruits and the instructor during the gasket 
making practice. 
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Figure 2-1. Demonstration of gasket construction procedure. 




Trouble Shooting 



The goal of the trouble shooting mini job situation was to test the recruit's 
ability to learn to perform elementary system analysis and trouble shooting on a 
hydraulic pressure system. A simulated pressure system was used as the appara- 
tus for both the training and testing aspects. After a taped introduction, the sub- 
jects were taught how the pressure system operates. Essentially, the apparatus 
consisted of a set of color coded and interchangeable gears which were so inter- 
connected that a simulated pumping system was driven. A schematic represen- 
tation of the system is shown in Figure 2-3. Within the simulated pumping system, 
a set of valves controlled the flow. Accordingly, to diagnose a fault in the system, 
the r ecruit needed to understand such elementary relationships as: (l)the effects 
of gear size on pump speed/ rate of flow, (2) the effects of direction of gear rotation 
on flow, (3) how large and small gears can intermesh to produce changes in output 
rate, and (4) the effects of valve and pump function on system operation. 

A series of simulated light indicators was used to indicate the adequacy of 
the flow at various portions of the simulated system. The task of the subject was to 
observe the light indicator, determine whether any out of tolerance conditions ex- 
isted, state the cause of the condition, and what should be done to remedy the out 
of tolerance condition, if any. 

After the operation of the simulated system had been explained, various 
malfunction situations were presented, and the recruits were taught what locations 
in the system needed adjustment in order to correct the problem and the cause of 
the malfunction. After the training, the subjects were presented with a number of 
practice trouble shooting problems. A typical problem was: 

1. low pressure light "on" 

2. pump speed indicator on "low ' 

3. bypass valve "open" 

4. Say: "THERE IS NO LEAK IN THE SYSTEM. WHAT HAS 
TO BE DONE TO FIX THIS? " 

For the practice problems, both the correct answer and the reason for it 
being correct were discussed. 

In the test situation, 12 problems were involved. The subjects circled one 
or two of nine numbers corresponding to nine possible malfunction causes. Fig- 
ure 2-4 shows one of the instructors explaining a detail in the pressure system. 
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POWER PUMP SPEED VALVE PRESSURE 




Figure 2-3. Schematic representation of pump system for trouble shooting test. 
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Figure 2-4. Instructor explaining function of simulated pressure system. 
Equipment Operation 

In the equipment operation mini job learning situation, the subjects were 
taught how to start up and shut down a motor and pump apparatus. The students 
were required to learn a 33 step procedure, including several safety precautions. 
Each class member was then given an opportunity to practice starting up and shutting 
down the apparatus. The equipment operation procedure (somewhat abbreviated) in- 
cludes the following: 

1. removal of oil rags from under motor 

2. checking oil and gasoline levels with dipsticks 

3. adding fuel and oil from fuel and oil storage tanks 

4. replacing lids on oil and fuel tanks 

5. checking to determine if valve is in bypass position 

6. plugging in battery socket 

7. turning ignition switch 

8. turning bypass valve on, then off 

9. turning off ignition 

10. cleanup of oil and fuel spills 




After practice, a checklist type performance test was administered. Scoring was 
completed while the recruit performed and was based on adherence to correct pro- 
cedures and observance of safety precautions. All relevant materials for the equip- 
ment operation test are listed in Appendix of this report. 

Figure 2-5 shows a group being instructed in equipment operation. Figure 
2-6 shows a sailor practicing on the equipment, and Figure 2-7 shows a subject be- 
ing tested. 
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Figure 2-6. Subject practicing on equipment 
operation apparatus. 
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Assembly 



In the assembly miniature learning situation, the recruits were taught how 
to assemble a gate valve from its component parts. First, a demonstration. of the 
correct assembly procedure was presented. The correct assembly procedure in- 
cludes: 



1. screwing packing nut to top of stem 

2. screwing gate to bottom of stem 

3. winding gate all the way up the stem 

4. screwing gate and stem assembly onto body of valve 

5. screwing handle onto top of stem with handle nut 

6. screwing on two 3/4" nipples 

7. checking to determine if parts are fitted tightly together 



This demonstration was followed by a short practice session in which the students 
were allowed to assemble the valve themselves. The instructor circulated among the 
students during this practice session and assisted each one, as required. After the 
practice, each subject was individually tested on his ability to assemble the valve. 
Again, scoring was through the checklist procedure. 

Figure 2-8 shows the instructor presenting the correct valve assembly pro- 
cedure. Figure 2-9 presents the practice session, and Figure 2-10 shows a student 
beginning the valve assembly test. All relevant test materials for the valve assembly 
task can be found in Appendix A of this report. 
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Figure 2-8. Instructor teaching valve 
assembly procedure. 





Figure 2-9. 



Two students practicing valve 
assembly. 
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A subject was judged to "pass" the miniature evaluation battery if he scored 
"average" or better on the trouble shooting test and "average" or better on two of the 
five remaining tests. Such subjects were assigned to a "probable successful" group 
and assigned to.a ship in the Fleet for work in the machinist mate rate. Subjects not 
achieving this level were assigned to a "probable fail" group and were similarly as- 
signed. Only Applied Psychological Services is aware of which recruits have been 
placed in each group. 

The logic for the choice of cut scores was that the trouble shooting mini job 
learning and test situations were largely cognitive in nature, while the remaining situ- 
ations largely involved learning manipulative procedures. Thus, those in the prob- 
ably successful group exhibited some cognitive as well as manipulative/procedural 
skill learning ability in the mini job learning situation. 

The scores of each recruit in the sample on the GCT, ARI, MECH, CLER, 
and SP tests of the normal Navy classification battery were provided by the Recruit 
Classification Center at the Great Lakes Naval Training Center. These data are 
presented in Chapter III of the present report and indicate fairly close equivalence 
between the white and the black groups on these tests. 



Instructors and Test Administrators 

Two instructors/ test administrators managed each training and evaluation 
session. One instructor/ test administrator was a 36 year old retired, blackNavy 
Chief Petty Officer. At the time of the present program, he was employed by the 
North Chicago school system, which generously granted him leave time for the pur- 
poses of this program. 

The second instructor/ test administrator was a 29 year old white psychologist. 
The assigned duties of the black Chief were: 

1. taping the introduction to each lesson 

2. conducting the training segment of each lesson 

3. administering the tests to black recruits 

This instructor was thoroughly trained in the content he was to present, the teaching 
methods he was to employ, and the test procedures prior to implementation of the 
present program. 
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The duties of the white psychologist were: 



1. organization and direction of the mini job sample learning 
and testing program 

2. training the black instructor/ test administrator 

3. assisting the black instructor/ test administrator in his 
lesson presentations 

4. testing the white recruits 

A black instructor/ test administrator was used, because it seemed that the 
black recruits would be more motivated when the proctor was a member of their own 
race. Recent research (Cole & Bruner, 1971) has shown that low achievement black 
students will perform at a more optimal level for a black proctor than fora white 
proctor. In addition, the education, personality, verbal inflection, and method of 
treating the subjects of the black instructor/ test administrator were of such a nature 
that he could easily be identified with and understood by the black recruits. 



Setting 

All of the mini job sample learning and testing sessions were held in a large 
classroom provided by the Machinist Mate School, Naval Training Center, Great 
Lakes. This classroom was equipped with 12 student desks and six worktables of 
various sizes. The lighting, temperature, ventilation, space, and privacy were 
considered optimal for this study. 

All research sessions began at 0720 hours. A one-hour lunch break was al- 
lowed between 1130 -and 1230 hours. Most sessions were completed by 1500 hours. 
Table 2-1 shows the approximate amount of training, practic'e, and testing time for 
each mini job sample training and testing situation. The situations required from 25 
to 65 minutes of total training, practice, and testing time. These times are well with- 
in the acceptable range of testing times currently used in educational and military set- 
tings. 



One half of the subjects were processed between 8 December and 16 December 
1971. The remaining half were trained and evaluated between 10 January 1972 and 21 
January 1972. 




* Based on a group of eight students with two proctors present 



Qu es tionnoi re 



A 36 item personal background questionnaire was constructed by Applied Psy- 
chological Services to measure various facets of cultural deprivation including: (a) 
need achievement, (b) home environment, (c) school environment, and (d) other demo- 
graphic variables. This questionnaire was administered to all the recruits in the mini 
job learning situation sample. It was considered that these cultural factors could con- 
ceivably moderate learning ability tothe extent that the correlations between the mini- 
ature evaluation test scores and the ultimate performance criteria would be lowered. 

In addition. Applied Psychological Services administered the same question- 
naire to a control group of Machinist Mate A School recruits, who had met or sur- 
passed the screening criteria for that school. 

In tervi ew 

After completion of the final mini job sample learning and testing situation, 
each recruit was interviewed by one of the instructor/ test administrators in order 
to obtain reactions to the entire training and testing program. Basically, the subjects 
were asked to compare the tests and training they received in the present program 
with other types of tests and training they received in the past. All subjects were en- 
couraged to respond freely and openly to the interviewer. The interview questions 
were constructed in a manner allowing quantitative and qualitative analysis of the inter- 
viewee responses. 
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In tercorrel ations 

The correlations among the scores of recruits on the six mini job sample 
learning tests were determined, along with the correlations among mini job sample 
learning scores and scores on the Navy classification tests. The intercorrelation 
matrices are shown for each racial group in Tables 3-1 and 3-2. Examination of 
these matrices indicates, rather conclusively, that the six miniature job learning 
tests are measuring factors which are quite unique from the factors measured by 
the Navy classification tests. These results are probably due to the fact that the 
Navy classification tests are heavily loaded in verbal and cognitive components, 
while the miniature job learning tests emphasize perception and performance. In 
addition, the correlations demonstrate that the miniature job learning tests are rela- 
tively independent from each other. Finally, we note that the single correlation co- 
efficient of any substantial magnitude, the correlationbetween the mechanical test of 
the Navy classification battery and the gasket cutting mini job learning test (for the 
white group only), might have been anticipated on the basis of the mechanical aspects 
of both of these tests. 



Means and Standard Deviations 

The means and standard deviations of the recruits sampled on the Navy classi- 
fication tests and on the miniature learning tests were also computed by racial group. 
The results are presented in Table 3-3. For the Navy classification tests, although 
the means and standard deviations of the white group are higher for all the tests, 
none of the mean .differences are statistically significant ("t" test). However, there 
is a small but statistically significant variance difference (p < .05, Fmaxtest) for 
the mechanical and the clerical tests of the Navy classification tests. Forbothof 
these tests, the variance of the white group was greater than that of the black group. 

This result supports a contention that the attempt to match the two groups on 
the basis of their Navy classification test scores was moderately successful. 

Of considerably more importance is the finding that for the miniature job 
sample tests there are no statistically significant differences between racial groups. 
This holds for both the mean differences and the variance differences. Accordingly, 
it seems that the mini job learning situational tests can beheldto possess minimum 
cultural bias. 
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Table 3-1 



Pearson Product Moment Correlations Between the GCT, ARI, MECH, 
CLER, SP, Equipment Use and Nomenclature (EUN), Gasket Cutting (GC), 
Meter Reading. (MR), Trouble Shooting (T), Equipment Operation (EO), and 
Assembly (A). Tests for 50, Low Aptitude, White, Naval Recruits 



Test 





ARI 


MECH CLER 


SP 


EUN 


GC 


MR 


T 


EO 


A 


GCT 


. 52 


-.15 .38 


. 07 


. 15 


-. 16 


-. 05 


-. 10 


-. 07 


. 13 


ARI 




.13 .41 


. 03 


.11 


.01 


. 25 


. 13 


. 14 


.08 


MECH 




.20 


. 30 


.21 


.64 


. 13 


.25 


. 19 


.01 


CLER 






-. 10 


. 15 


.05 


.00 


-. 04 


-. 15 


-.06 


SP 








. 22 


.26 


-. 08 


. 07 


.24 


.03 


EUN 










. 18 


. 14 


-. 06 


-. 06 


-.09 


GC 












.09 


. 29 


. 23 


.01 


MR 














.27 


-. 09 


-. 17 


T 
















. 38 


. 16 


EO 






* 












. 08 



Table 3-2 

Pearson Product Moment Correlations Between the GCT, ARI, MECH, 
CLER, SP, Equipment Use and Nomenclature (EUN), Gasket Cutting (GC), 
Meter Reading (MR), Trouble Shooting (T), Equipment Operation (EO), and 
Assembly (A) Tests for 49, Low Aptitude, Black, Naval Recruits 



Test 





ARI 


MECH 


CLER 


SP 


EUN 


GC 


MR 


T 


EO 


A 


GCT 


. 20 


. 16 


.29 


. 22 


. 09 


-.25 


. 20 


. 14 


. 09 


. 10 


ARI 




. 37 


.24 


. 36 


-. 19 


.23 


. 06 


. 25 


. 02 


. 16 


MECH 






. 01 


. 31 


-. 11 


-.04 


. 23 


. 16 


-.04 


-. 02 


CLER 








. 36 


. 14 


-.21 


. 15 


. 11 


. 06 


.05 


SP 










. 05 


-. 14 


. 13 


. 27 


-. 09 


. 19 


EUN 












-. 30 


-.09 


-.23 


. 13 


. 04 


GC 














-.20 


-. 08 


. 02 


. 19 


MR 
















. 31 


. 14 


-. 17 


T 
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Table 3-3 



Means and Standard Deviations on the GCT, ARI, MECH, CLER, SP, Equip- 



ment Use and Nomenclature (EUN), Gasket Cutting (GC), Meter Reading 
(MR), Trouble Shooting (T), Equipment Operation (EO), and Assembly (A) 
Tests for 50 White and 49 Black, Low Aptitude, Naval Recruits 


Test 


White 


Black 


Mean 


S.D. 


Mean 


S.D. 


GCT 


39. 58 


7. 33 


38.29 


6. 52 


ARI 


42. 56 


5. 83 


41.24 


4.27 


MECH 


42. 16 


6. 22 


40.73 


4.54 


CLER 


51. 90 


12. 27 


51. 82 


8.68 


SP 


43. 44 


5. 30 


41.51 


6. 00 


EUN 


21. 16 


3. 17 


20.59 


2. 60 


GC 


14. 50 


2. 78 


14.31 


2. 78 


MR 


10. 92 


4. 26 


10.78 


4. 34 


T 


14. 60 


5. 64 


13. 82 


5.56 


EO 


59. 78 


7. 02 


57.88 


6.56 


A 


22. 84 


3. 61 


23.29 


3. 25 
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"Probably Acceptable"- "Probably Unacceptable" 

As stated previously* a recruit who scored at or above the group mean on 
the miniature job learning trouble shooting test and who scored similarly on any two 
of the remaining five miniature job learning tests was placed in a group for which 
"probably acceptable" Fleet performance is predicted. It is anticipated that the 
members of this grrmp will demonstrate at least minimally satisfactory progress on 
the machinist mate job in the Fleet. On this basis, 60 per cent of the white recruits 
in our sample (n = 30) and 53 per cent of the black subjects (n = 26) were placed in 
the "probably acceptable" category. A chi-square analysis was performed on these 
data. The results of the chi-square analysis indicated that race was not significantly 
associated with "passing" or "failing" the miniature evaluation battery. This result 
was expected, inasmuch as the mean difference across race for each test separately 
was not statistically significant. 



Reliability 

The test administration procedures for the equipment operation, assembly, 
and gasket cutting tests permitted an analysis of interrater reliability. For these 
tests, a sample of the recruits was scored independently by both the black instructor/ 
test administrator and by his white counterpart. The separate scores, so deter- 
mined, were compared. The results of this interrater reliability analysis are shown 
in Table 3-4. Examination of Table 3-4 shows that the interrater reliability coeffi- 
cients were acceptably high for performance tests of this type. In addition, the 
means and standard deviations across procters were almost identical. This suggests 
that scoring methods for the procedural tests were sufficiently objective to allow a 
reliable total score estimate. 

Table 3-4 



Means, Standard Deviations, and Interrater Reliability Coefficients for a Black 
Test Administrator and a White Test Administrator Scoring the Gasket Fabrica 
tion(GC), Equipment Operation (EO), and Assembly Tests (A) 







GC 


Test 


EO 




A 






Administrator White 




Black 


White 




Black 


White 


Black 


n 




32 






39 




39 




Mean 


14.41 




14.72 


56.03 




55.69 


22.05 


22.26 


S.D. 


2.23 




3. 10 


7.49 




7.46 


3.21 


3. 33 


r 




.75 


# ' 




.97 




. 96 
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The Interview 



After completion of his learning and evaluation session, each examinee was 
interviewed by the instructor/ test administrator of the race of the examinee. The 
four questions involved in each interview were; 

1. How would you compare the training portion of this program 
with other training programs you have encountered in the 
past? Was it better, worse, or about the same? Why? 

2. How would you compare the tests you took today with the 
more traditional paper and pencil tests you have takenin 
school? Are they better, worse, or about the same? Why? 

3. Did you enjoy participating in today's program? Why? 

4. Please tell me any other impressions or thoughts you have 
about the training program. 



Seventy-eight per cent of the recruits thought the training portion of the pro- 
gram was better than the training they had received in other programs. Twenty per 
cent of the subjects thought the training portion of the program was the same as the 
training they had received in other training programs. Only one per cent of the sub- 
jects, though, thought the training they received was worse than that of other training 
programs.. These results support the emphasis placed on performance during the 
learning sessions rather than on reading and writing. Some representative subject 
responses to this question were: 






* 
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1. The training was easy because it doesn't involve reading. 
Never took one (training program) like it before. I'm at 
the fourth or fifth grade in reading. 

2. Had someone explain it to you step by step. Gave you 
time to do it. 

3. Got more out of it. You learn more. 

4. Mechanical stuff is my type of work. It's my thing. I 
dig working with my hands. 

5. You get more out of putting things together than in look- 
ing at pictures and directions. You can see what each 
part looks like, but you can't in a picture. If you have 
it in your hand you know what it feels like. 
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With regard to the second question, 86 per cent of the subjects thought the 
miniature job learning tests were "better than" paper and pencil tests. Twelve per 
cent of the subjects thought the program tests were the "same as" paper and pencil 
tests. Again, only one per cent thought the program tests were "worse than" paper 
and pencil tests. These results support the use of performance oriented tests which 
require little or no reading. Sample interview responses to this item are: 

1. You don't have to do as much writing. In school they 
made me more tense with more pressure on my mind. 

2. Because you see what you're doing. In school tests 
you have to read it from a book. You have to keep it 
in your mind. You don't get to see what you're doing. 

3. The questions don't drag on. Can't understand the ones 
in school as well. 

4. Just asked the question right after showed howto do it. 

5. Because you don't spend the time reading the questions. 

6. When you read it yourself, you might not understand the 
words, but when he read it out you don't get a chance to 
goof up on the words. 

The responses to the final question showed that 98 per cent of the subjects 
enjoyed the training and evaluation program, while only two per cent did not enjoy 
it. These results and the results of the first two questions allow the conclusion that 
"low aptitude" Naval Recruits exhibit an overwhelming attitudinal preferencefor the 
learning and evaluation program, as here employe^ over the more traditional testing 
programs. 



The Questionnaire 

The primary purpose of the cultural deprivation questionnaire in the current 
investigation is for eventual application as a statistical control. Before the question- 
naire is used as a control measure, though, its structure must be established. 
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Factor Analysis 



The questionnaire scores for the entire sample were subjected to a principal 
components factor analysis with a varimax rotation. Nine factors, accounting for 
46 per cent of the predictable variance, were extracted. These were called; self-es- 
teem, environmental stimulation, reading habits, educational attainment, educational 
initiative, parental interest, monetary deprivation, educational encouragement, and 
urbanity. The items with heaviest loadings on each of the nine extracted factors are 
presented in Tables 3-5 through 3-13. 



Table 3-5 

Items with Highest Loadings on Self-Esteem* Factor 

Item Loading 

At this time, what do you think your chances are of successfully 
advancing in the Navy are? .757 

At this time, what do you think your chances are of successfully 
passing the tests for one or more of the Navy rating(s) you hope 
to enter? • ?30 

Approximately what yearly salary do you think you will be earning 
ten years from now? • 587 



During your past schooling, how would you have done in school if 
you had done the very best you could? 

Comparing yourself to others you know, how do your decisions 
seem to stack up in quality? 

How did you compare with other fellows in rate of progress 
through school? 



•. 478 
-.476 
-. 335 



k 



♦Eigenvalue = 2.685 



I 

f; 

iv 

i 

r. 



41 



50 



Table 3-6 
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Items with Highest Loadings on Environmental Stimulation* Factor 



Item Loading 

How many rooms did your home have when you were a child? . 657 

How much education did your mother have? . 582 

How much education did your father have? . 565 

How many books do you now own? . 420 

To how many magazines and periodicals did your family 

subscribe whileyou were growing up? . 388 

When you were growing up how many books were around the 

house? -.635 

How do you feel about the achievements of your parents? 356 



*Eigen value = 2. 302 

Table 3-7 

Items with Highest Loadings on Reading Habits* Factor 



Item Loading 

Other than schoolwork, how much reading did you do during 

your youth? .767 

During your school years, to what extent did you read news- 
papers? .608 

About how often do you spend an evening at home sitting around 

and reading? -.650 



*Eigen value = 2. 038 
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Table 3-8 

Items with Highest Loadings on Educational Attainment* Factor 



Item Loading 

How much education have you had? . 542 

How many serious, nonfiction books have you read in the past 

year, not counting text books? . 480 

How did you compare with other fellows in rate of progress 

through school? -.567 

How difficult was high school work for you? -. 489 

How did your parents feel about the marks you made in school? -. 424 

During your past schooling, how would you have done in school 

if you had done the very best you could? -.318 

How often did you seriously consider quitting school? -. 313 



* Eigen value = 1.791 

Table 3-9 

Items with Highest Loadings on Educational Initiative* Factor 



Item . Loading 





To how many magazines and periodicals did your family 
subscribe while you were growing up? 


. 382 




How often did you seriously consider quitting school? 


-.747 


As you grew up bow did you feel about school? 


-. 684 


i 

r • 


How did you compare with your friends in rate of progress 
through school? 


-. 307 


r 

% 


* Eigen value = 1.772 
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Table 3-10 



Items with Highest Loadings on Parental Interest* Factor 

Item Loading 

While you were in school, how much interest did your parents 
or guardians appear to take in your school work? .654 

During most of your school years, would you say that your needs 
were; [(a) well provided for; (b) satisfactorily provided for; (c) 
somewhat meagerly provided for but tolerable; (d) unsatisfied 
most of the time; (e) never satisfied]? .591 

How did you feel about the achievements of your parents? .526 

How did your parents feel about the marks you made in school? .376 

Eigenvalue = 1.625 

Table 3-11 

Items with Highest Loadings on Monetary Deprivation* Factor 

Item Loading 

When you were a child, did your parents talk or act as though 

money were a problem? .652 

During your high school years, what was your total family income 
per month? .570 

Dun'sti{ ibost of ybur school years, would you say that your needs 
were? |<a) well provided for; (b) satisfactorily provided for; (c) 
somewhat meagerly provided for but tolerable; (d) unsatisfied 
most of the time; (e) never satisfied]? -.438 



Eigen value * 1.491 



Table 3-12 



Items with Highest Loadings on Educational Encouragement* Factor 

Item Loading 

When you were a child, did your parents compare your school 

performance (favorably or unfavorably) with that of other 

children? . 597 

When you were a child, did your parents sometimes tell you to 

stay inside and read more? . 568 

As a child, how often did your parents encourage you to read? .429 

*Eigen value = 1.387 

Table 3-13 

Items with Highest Loadings on Urbanity* Factor 

Item Loading 

How would you describe the neighborhood in which you were 

brought up (degree of crowding)? .502 

The place in which you spent the most time during your early 

life was a (city size)? -.671 



’•''Eigen value = 1. 367 



Significance Tests 

In order to test whether or not the "low aptitude" group differed significantly 
on the cultural deprivation factors from a group which meets the selection stand- 
ards for the machinist mate "A" school, the cultural deprivation questionnaire was 
administered to 118 recruits in the "A" school. The factor score means for the 
"high aptitude" (A school) group and for the "low aptitude" group were calculated. 
These are presented in Table 3-14. Tests ("t" tests) were conducted between the 
group mean scores for each factor. The results of these tests are also presented 
in Table 3-14. For seven of the nine factors, the "low aptitude" group demon- 
strated significantly more cultural deprivation, as measured by this questionnaire, 
than the "high aptitude" group. Accordingly, cultural deprivation (as here meas- 
ured) and "low aptitude" appear to go hand in hand. 

Examination of the data in Table 3-14 indicates no statistically significant dif- 
ferences across aptitude groups in the parental interest factor. 

In another factor, educational encouragement, the "high aptitude" group seems 
to be more deprived than the "low aptitude" group. It is quite possible that the 
"low aptitude" group perceives educational encouragement differently than the "high 
aptitude" recruits. "Low aptitude" persons may perceive educational encourage- 
ment as nagging. The young deprived child then would associate academic pursuits 
with nagging and parental disfavor, resulting in discouragement of the child's aca- 
demic efforts. An alternate explanation is that the "low aptitude" group receives 
more educational encouragement because they tend to perform less well in school. 
A child who is already performing well does not need as much encouragement as 
the child who is performing poorly. 



Correlation with Tests 



One of our hypotheseswas that the miniature job evaluation tests would be 
less contaminated by cultural deprivation factors than the usual Navy qualification 
tests. The. Pearson product moment correlation among the qualification test scores, 
the miniature job sample tests, and the questionnaire factor scores for the 99 re- 
cruits in the "low aptitude" sample are presented in Table 3-15. In general, the 
correlations between the factor scores and the test scores tend to be low for both 
the Navy qualification test and the miniature job learning tests. However, only one 
of the correlations between the mini learning tests and the deprivation factors is 
statistically significant. On the other hand, seven of the correlations between the 
Navy qualification tests and the cultural deprivation factors scores are statistically 
significant. The data in Table 3-15, then, support our contentions that the mini 
tests are unrelated to cultural factors and are less culturally loaded than the usual 
Navy qualification tests. 
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Table 3-14 



Means, t-Ratios, and Significance Levels for "High Aptitude" 
and "Low Aptitude" Recruits on Nine Factors 

High Aptitude Low Aptitude 
(n = 118) (n=99) 





Mean* 


Mean* 


t- Ratio 


Self-Esteem 


-147. 40 


-5.90 


8.51 (p<. 001) 


Environmental Stimulation 


-34.58 


2.20 


3.52 (p<. 002) 


Reading Habits 


-21. 67 


-2.30 


2.69 (p<. 01) 


Educational Attainment 


-91. 64 


-0.39 


8.71 (p<. 001) 


Educational Initiative 


-37. 87 


1.52 


3.00 (p<. 01) 


Parental Interest 


6. 34 


-3.69 


0.55 (N. S. ) 


Monetary Deprivation 


-32. 37 


-13.36 


2. 17 (p<. 05) 


Educational Encouragement 


15.29 


-0. 18 


<3.43 (p<. 002) 


Urbanity 


-10. 86 


0.22 


3.32 (p<. 002) 



* Lower scores indicate less cultural deprivation. 
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CHAPTER IV 



C R ITER ION DEVELOPMENT 



Quite obviously, the demonstrated reliability, freedom from cultural bias, 
equivalence for both white and black groups, and the like represent necessary but 
not sufficient ingredients for an assessment approach. The predictive validity of 
the miniature job learning tests remains to be demonstrated. To this end, each 
of the recruits in our "low aptitude" sample will be followed up after he has served 
6 months, 12 months, and 24 months in the machinist mate rate. The longitudinal 
followups will be based on a criterion referenced test approach, supplemented by 
supervisor ratings and supervisory interview data. We place little, if any, cre- 
dence in the supervisory ratings and interview data since it is known that super- 
visors often rate black job incumbents lower than whites. Flaugher, Campbell, 
and Pike (1969) found that white supervisors rated a group of Negro incumbents one- 
half a standard deviation lower than Negro supervisors. Negro supervisors, though, 
did not rate white incumbents higher or lower than white supervisors. 

Nevertheless, supervisory evaluative data are considered to be of interest 
in the present context. Moreover, the black and the white "low aptitude" recruits 
have been equated on both the usual Navy tests and the miniature job learning tests. 
The followup criterion referenced performance data will also be available for both 
racial groups. Accordingly, the opportunity will exist for checking this response 
bias conjecture, found in industry, in the Navy context. 
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Criterion Te. sis 

To develop the criterion tests for use in this study. Applied Psychologi- 
cal Services’ personnel reviewed first portions of NAVPERS 18068A relevant to 
the machinist mate rate. Several proposed ideas for criterion referenced six 
month Fleet performance tests were extracted. The next step was to elaborate 
on these testing possibilities with experts in the machinist mate rate. The ex- 
perts in this situation were primarily needed to supply scorable testing items, 
fruitful testing suggestions, and statements of criteria for "acceptable" work on 
each criterion objective after six months experience as a machinist mate striker. 
Four such experts were made available by the Naval Damage Control Training 
Center, Philadelphia. 



$ Three Chief *>rnnt Officers end One Hester Chief Petty Officer. 
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These experts agreed that the following practical performance items would 
constitute an adequate test of the ability of a machinist mate striker, with six 
months Fleet experience, to meet normal performance expectations: 



1. standing messenger watch 

2. making and breaking a flange 

3. packing a valve 

4. procedures in common malfunction and in emergency 
situations 

5. knowledge of use and names of common equipment 
and tools 

6. general alertness and common sense in the work situ- 
ations 

Three separate meetings were held to isolate, derive, and define the per- 
formance objectives and the methods for measuring performance on these objec- 
tives. As a result, six nonverbal performance tests were defined. Each of these is 
described categorically below. 



Me ssage Valrh 

The message watch examination is a test of the. examinee's ability to record 
data accurately from a throttle board and to determine malfunctions indicated by the 
data. Pictorial and simulated throttle board situations are presented, oneata time, 
to the examL. 2 e. The examinee is required to record accurately data from the 
throttle board pictures and to report to the examiner those throttle board readings 
which indicate a malfunction in the system. The examinee receives two scores. One 
score is based on his degree of accuracy in recording data from the simulated 
throttle board picture; the second score indicates his ability to detect malfunctions 
from the data. A sample item is presented as Figure 4-1. 



V 
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Figure 4-1. Sample throttle board test item. 



Making-Breaking a Flange 

The making-breaking a flange test is an individually administered perform- 
ance test. The examinee is required to break and make a flange using the following 
tools and items; (a) a gasket, (b) one assembled six inch flange, (c)one scraper, 

(d) two box end wrenches, and (e) one rag. The flange on which the test is perform- 
ed is shown in Figure 4-2. Scoring is based on following the correct procedures, 
care and use of tools, and adherence to safety regulations. 




Figure 4-2. Flange for flange making-breaking test. 
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Packing a Valve 



The valve packing test is also an individually administered performance 
test. The examinee is required to pack a valve using: (a) a large, mounted valve, 
(b) packing material, (c) a knife, (d) box end wrenches, (e) a packing puller, and 
(f) one very large adjustable wrench. Scoring is based on following the correct pro 
cedures, care and use of tools, and adherence to safety regulations. 
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Malfunction and Emergency Procedures 

The malfunction and emergency procedures test is an individually adminis- 
tered test. Each item in the test consists of a set of pictures depicting a common 
emergency or malfunctive correction sequence. The task of the examinee is to place 
the pictures, which are presented in scrambled order, in the correct sequence. To 
do this the examinee must, of course, first recognize what is being represented. 
Each item in the set consists of a situation which is critical to or frequently encoun- 
tered in machinist mate performance. The various problems depicted (in order of 
difficulty) in the picture arrangement test are; (a) electric shock, (b) fire in com- 
partment, (c) leaky valve repair, (d) tank gauging, (e) fire hose assembly, (f) rup- 
tured pipe, and (g) spring bearing repair. 

Scoring is based on the number of pictures in each item set which are placed 
in the correct order. A time bonus is given for fast performance. The number of 
pictures in each set ranges from four to six. A sample item is presented in Figure 
4-3. 



Equ i pro en t /Too 1 s Names and Use 

The equipment/ tools names and use test is also individually administered. 
The items consist of a set of cards. Each card shows a typical machinist mate 
work situation and three tools which might be employed to complete the job. The 
items are arranged in a hierarchical order of difficulty. For each item, the exam- 
inee is required to select, from among the three shown, the best tool for completing 
the specific job depicted. Extra credit is awarded if the examinee can also state 
verbally the correct name of the tool. A sample item is shown in Figure 4-4. 
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Figure 4-4. Sample equipment/ tools names and use item. 



General A1 cr tness/Common Sense 



The general alertness/common sense test is also a pictorial, individually 
administered test. Each item consists of a picture of a typical machinist mate 
work situation in which the sailor shown is doing something wrong. The task of the 
examinee is to detect and report what is wrong in each picture. Theitemsare ar- 
ranged in ascending order of difficulty and the scoring isbased on the number of 
correct responses. A sample item is shown in Figure 4-5. 



Validation of Miniature Job Sample Learning Tests 

Once the criterion data are collected. Applied Psychological Services will 
determine the extent that the miniature job sample learning tests predict the cri- 
terion scores. The working hypothesis is that the miniature joblearning tests, on 
a collective basis, will predict criterion performance better than the usual Naval 
classification tests for both the white and the black "low aptitude" recruits. We also 
hypothesize that no differential validity will be involved and that significantly differ- 
ent criterion test scores are anticipated for our "probably acceptable" and the 
"probably unacceptable" groups. As stated previously, supervisor evaluative data 
will be collected, and supervisory interviews will be conducted. These interview 
and rating data will provide further insight into the ability of those who have passed 
the miniature job learning tests but who scored below the cut point for the machinist 
mate rating on the usual Navy classification tests. Finally, the criterion tests will 
be administered to a sample of machinist mates who possess the same experience 
in the Fleet as our "low aptitude" sample but who have graduated from the machin- 
ist mate "A" school. 

The end product will be a method for identifying those persons who can be 
successful in the Fleet who might otherwise be eliminated from consideration. The 
results may suggesta dualistic testing approach in whichpersons scoringbelow 
cut points on the usual Navy tests are given a "second chance" through a battery of 
nonverbal miniature job learning tests. 
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Figure 4-5. Sample general alertness/ common 
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CHAPTER V 



SUMMARY AND CONCLUSIONS 
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This study was designed to develop a nonverbal, culture fair assessment pro- 
cedure which identifies Navy recruits who can learn to perform Navy jobs in skilled 
ratings even though they fail to meet the usual selection standards as measured by 
the current Navy classification tests. The underlying hypothesis was that recruits, 
judged by the usual classification methods to be of "low aptitude" but who exhibit the 
ability to learn and perform sample aspects of a Navy job, will be able to learn and 
perform on the total job--provided proper on-the-job training is given. 

The machinist mate rating was used as a basis for testing the theoretic and 
methodological concepts involved. A sample of 50 white and 49 black "low apti- 
tude" recruits took part in a series of miniature job learning and evaluative situa- 
tions. These situations sampled those jobs which are frequently performed by ma- 
chinist mates during their first six months in the Fleet or which are critical to 
performance during this period. At the conclusion of each job learning session, a 
skill based, nonverbal performance test was administered to each recruit. The re- 
cruits also; (1) completed a personal background questionnaire, and (2) participated 
in a post-test interview which inquired into reactions to the training and testing meth- 
ods involved. 

The personal background questionnaire was also administered to 118 persons 
who met the usual qualifying standards for the machinist mate rate. Those "low 
aptitude" recruits, to whom the miniature job learning instructional tests were ad- 
ministered, have been placed aboard ships of the Atlantic Fleet for entry into the 
machinist mate rating. 

A battery of criterion referenced performance tests has been developed for 
measuring the level of Fleet achievement of those "low aptitude" persons who, on 
the basis of the miniature job sample learning test results, were classified as 
"probable acceptable" or "probable unacceptable" in terms of predicted Fleet per- 
formance. 

• The scores of the 'low aptitude" sampleon the miniature job learning tests, 
along with the scores of the sample on the usual Navy classification tests were inter- 
correlated. These data, along with interscorer reliability data and measures of cen- 
tral tendency, were used as the primary basis for preliminary evaluation of the min- 
iature job learning tests. 
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The personal history questionnaire was factor analyzed, and the factor scores 
of both the "low aptitude" and the "A" school sample were compared. Additionally, 
the factor scores of the "low aptitude" sample were correlated with their scores on 
the mini learning tests and on the usual Navy classification tests. The results of the 
work, completed to date, suggest the following conclusions; 

1. The six miniature job learning tests measure factors 
which are independent from the factors measured by the 
usual Naval classification tests. 

2. The white group and the black group performed equally 
well on the miniature job learning tests; accordingly, 
the tests are considered to be reasonably "culture fair. " 

3. Adequate interscorer reliability was demonstrated. 

4. Statistically significant differences were evidenced be- 
tween the factor scores of the "low aptitude" sample and 
the factor scores of the "A" school sample on eight of the 
nine cultural deprivation factors extracted from the cul- 
tural deprivation questionnaire. 

5. The mini job learning tests are less culturally loaded 
than the usual Navy classification tests. 

6. The type of testing program here described is preferred 
by "low aptitude" personnel over the usual type of classi- 
fication testing performed in the Navy. 
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LESSON IV 



Training Evaluation 
Equipment Operation 
Examiner Instructions 



Task - This is a test of the examinee's ability to start up and shut down a 
motor. 

Test Materials - The following materials should be provided; 

1. Two electric motors 

2. Two long dipsticks 

3. Two short dipsticks 

4. Two rags 

5. Two oil cans filled with red water 

6. Two fuel cans filled with green water 

7. Two battery boxes with extension cords 

8. Scoring checklist for each student. Make sure the student's 
name is on the checklist before you begin testing him. 

9. Two stopwatches 

General Directions to Examiner; 

1. Each examiner will be tested individually in a quiet, well lighted 
room or area. 

2. All students, other than the examinee, must be kept outside of 
the test room while testing is in progress. 

3. Read very carefully and thoroughly the "Examinee Instructions. " 
Be certain you understand the test and the method for administer- 
ing it before attempting to give it to the examinee. 

4. Make certain each examinee understands his instructions before 
he begins the test. Supply no information beyond what is needed 
for understanding the test procedure. Remember, this is a test 
situation, not a training one. 

5. Make certain that all examinees who have eyeglasses are wearing 
them. 

6. Allow each subject 10 minutes to start up and shut down the motor. 

7. After you have finished testing one subject, he is to remain quietly 
in the testing room out of view (behind) the next testee. 
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Directions to Examinees (to be read on tape) 

"THIS IS A TEST OF YOUR ABILITY TO START UP AND SHUT DOWN 
A MOTOR. MACHINIST MATES PERFORM TASKS VERY MUCH LIKE THIS. " 

"I WANT YOU TO START UP AND SHUT DOWN THIS MOTOR IN EX- 
ACTLY THE SAME WAY AS YOU WERE TAUGHT. TRY TO PERFORM EVERY 
STEP EXACTLY AS WE SHOWED YOU DURING THE LESSON. YOU WILL ONLY 
RECEIVE FULL CREDIT IF YOU PERFORM THE STEPS CORRECTLY AND IN 
THE RIGHT ORDER. IF YOU DO NOT PERFORM THE STEPS IN THE RIGHT 
ORDER, YOU WILL NOT RECEIVE FULL CREDIT. " 

"YOU WILL HAVE TEN MINUTES TO COMPLETE THIS TASK. " 

"BEGIN. " 



Scoring of Equipment Operation Checklist 

1. Place a yes after each item that is performed correctly. 

2. Place a after each item that is performed incorrectly. 

3. Leave the space blank if the step is not performed. 

4. Encircle each correctly performed but out of sequence step. 

5; Allow two points for each correctly performed step. 

6. Allow one point for each correctly performed (circled) step out of 
sequence, except for reversals of screwing on the oil and fuel tank 
lids. 

7. Do not allow any points for incorrect steps or for steps not performed. 



A-2 






i 77 



LESSON IV 



Training Procedure for Equipment Operation 



To the Instructor: Keep this sheet in front of you at all times during this 
training session. Do and say exactly what is said on these pages. 

Procedure 




1. Stand next to the motor. 

2. Say: "I HAVE IN FRONT OF ME A SMALL MOTOR WITH A PUMP AT- 
TACHED TO IT. THERE ARE SEVERAL STEPS I MUST PERFORM IN 
ORDER TO START UP THIS MOTOR. " 

3. Say: "MY FIRST STEP IS TO REMOVE ANY OILY RAGS OR TRASH 
LYING ON OR NEAR THE MOTOR. YOU CAN ALL SEE THAT THERE 
IS A RAG NEXT TO THE MOTOR. I WILL REMOVE IT. IF I DIDN'T 
REMOVE THE RAG, THERE IS A CHANCE THAT A SPARK FROM THE 
MOTOR WOULD IGNITE THE RAG AND CAUSE A FIRE. " 

4. Remove rag. 

5. Say: "THE SECOND STEP IS TO CHECK THE FUEL LEVEL USING A 
LONG DIPSTICK. REMEMBER TO USE THE LONG DIPSTICK WHEN 
YOU CHECK THE FUEL LEVEL. " 

6. Hold up long dipstick. 

7. Say: "THIS IS THE FUEL TANK. " 

8. Point to fuel tank. 

9. Say: "I WILL UNSCREW THE FUEL TANK LID, CLEAN OFF THE DIP- 
STICK, AND PUT THE DIPSTICK IN UNTIL IT TOUCHES THE BOTTOM 
OF THE FUEL TANK. " 

10. Unscrew top of fuel tank, clean the dipstick, and put long dipstick in until 
it touches bottom. Pull dipstick out of fuel tank and hold it up to the class. 
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11. Say: "NOTICE THAT THE FUEL LEVEL IN THE FUEL TANK IS NOT 
FULL. THE FUEL LEVEL MUST REACH THIS WHITE LINE TO BE 
FULL. " 

12. Point to white line. 

13. Say: "SINCE THE FUEL TANK IS NOT FULL, I WILL ADD FUEL FROM 
THE FUEL STORAGE TANK WITH THIS FUEL FUNNEL. " 

14. Hold up fuel funnel to class. Point to fuel storage tank. Insert funnel in 
fuel tank. Unscrew fuel storage tank. Pour fuel from fuel storage tank 
into fuel tank. Put fuel storage tank aside. Put lid on fuel storage tank. 

15. Say: "REMEMBER THAT AFTER YOU POUR FUEL YOU MUST SCREW 
THE LID BACK ONTO THE FUEL STORAGE TANK. NOW I WILL AGAIN 
CHECK THE FUEL LEVEL WITH THE DIPSTICK. BEFORE I CHECK THE 
FUEL LEVEL, I MUST WIPE OFF THE DIPSTICK WITH A RAG. " 

16. Check fuel level with long dipstick, after wiping it off with a rag. 

17. Say: "YOU CAN NOW SEE THAT THE FUEL LEVEL HAS REACHED THE 
WHITE LINE. MY NEXT STEP IS TO PUT THE LID BACK ONTO THE 
FUEL TANK. " 

18. Put lid back on fuel tanks. 

19. Say: "NOW I MUST SEE IF THE OIL LEVEL IS CORRECT. I WILL CHECK 
THE OIL LEVEL IN THE SAME WAY AS I CHECKED THE FUEL LEVEL 
NOTICE THAT I USE A SHORT DIPSTICK RATHER THAN A LONG DIPSTICK, 
AN OIL STORAGE CAN RATHER THAN A FUEL STORAGE CAN, AND AN 
OIL FUNNEL RATHER THAN A FUEL FUNNEL. " 

20. Point to these three items and mention their names again: "SHORT DIPSTICK, 
OIL STORAGE CAN, AND OIL FUNNEL. " 

21. Say: "THE OIL IS NEEDED TO LUBRICATE THE MOTOR AND THE PUMP. " 

22. Say: "NOW I WILL CHECK THE OIL LEVEL USING THE SHORT DIPSTICK. " 

23. Hold up short dipstick. 

24. Say: "THIS IS THE OIL TANK. " 
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25. Point to oil tank. 
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26. Say: "I UNSCREW THE OIL TANK LID, WIPE OFF THE DIPSTICK, 

AND PUT THE DIPSTICK IN UNTIL IT TOUCHES THE BOTTOM OF 
THE OIL TANK. " 

27. Unscrew top of oil tank and put short dipstick in until it touches bojttom. 

Pull dipstick out of oil tank and hold it up to the class. 

28. Say: "NOTICE THAT THE OIL LEVEL IN THE OIL TANK IS LOW. THE 
OIL LEVEL MUST REACH THIS WHITE LINE. " 

29. POINT TO WHITE LINE. 

30. Say: "SINCE THE OIL TANK IS NOT FULL, I WILL ADD OIL FROM THE 
OIL STORAGE TANK WITH THIS OIL FUNNEL. " 

31. Hold up oil funnel to class. Point to oil storage tank. Insert funnel in 
oil tank. Unscrew oil storage tame. Pour oil from oil storage tank into 
oil tank. Put oil storage tank ; side. Put lid on oil storage tank. 

32. Say: "REMEMBER, AFTER YOU POUR OIL YOU MUST SCREW THE LID 
BACK ONTO THE OIL STORAGE TANK. NOW, I AGAIN CHECK THE OIL 
LEVEL WITH THE DIPSTICK. BEFORE CHECKING THE OIL LEVEL, I 
AGAIN WIPE THE DIPSTICK OFF WITH A RAG. " 

33. Check oil level with short dipstick. 

34. Say: "YOU CAN NOW SEE THAT THE OIL LEVEL HAS REACHED THE 
WHITE LINE. MY NEXT STEP IS TO PUT THE LID BACK ONTO THE 
OIL TANK. " 

35. Put lid back on oil tank. ^ 

36. Say: "NEXT I WILL CHECK TO SEE IF THIS VALVE IS IN THE BYPASS 
POSITION. " 

37. Point to valve. 

38. Say: "THE VALVE IS NOW IN THE BYPASS POSITION, IF NO WATER 
IS FLOWING INTO THE BUCKET. " 

39. Point to inside of bucket. 

40. Say: "NEXT, WE WILL CONNECT THE BATTERY WITH THIS PLUG. THIS 
IS THE BATTERY AND THIS IS THE PLUG. " 
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41. Point to battery and plug. Put plug into battery socket. 

42. Say; "NOTICE THAT I PUT THE PLUG INTO THE BATTERY JUST AS IF 
I WERE PLUGGING IN A LAMP. " 

43. Say; "NOW I WILL TURN ON THE IGNITION SWITCH. THIS IS THE IG- 
NITION SWITCH. " 

44. Point to ignition switch and turn it on. 

45. Say: "MY FINAL STEP IS TO TURN THE BYPASS VALVE ONTO THE PUMP. " 

46. Turn bypass valve so that water pumps. 

47. Say: "NOW WE MUST LEARN HOW TO SHUT THIS MOTOR OFF. ALL WE 
HAVE TO DO IS TURN THE VALVE BACK TO THE BYPASS POSITION. NO 
WATER SHOULD BE GOING INTO THE BUCKET. " 

48. Turn valve to bypass. 

49. Say; "AND TURN THE IGNITION OFF. " 

50. Turn off ignition. 

51. Say; "YOUR FINAL STEP IS TO WIPE OFF THE FUNNELS WITH A RAG 
. AND WIPE UP ANY OIL AND FUEL SPILLS THAT OCCURRED. " 

52. Say; "WE WILL GO OVER THIS ONCE AGAIN BEFORE WE ALLOW YOU 
TO PRACTICE WHAT YOU HAVE LEARNED. 

53. Empty fluid from oil and fuel tanks back into their storage containers. 

Perform steps 1-52 again. 

54. Allow each man 10 minutes to practice starting up (the other students can 
watch) and shutting off motor. Observe each man closely. Help them or 
prompt them whenever they are having difficulty. Correct mistakes. 
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LESSON IV 



Scoring Checklist for Equipment Operation 



Name 

1. Removes rag from under motor 

2. Unscrews fuel tank lid 

3. Wipes dipstick clean 

4. Inserts long dipstick 

5. Observes need for fuel 

6. Does not wipe dipstick 

7. Adds fuel from fuel storage tank 

8. with fuel funnel 

9. Cleans long dipstick 

10. Inserts long dipstick again 

11. Observes adequate fuel level 

12. Does not wipe dipstick 

13. Puts lid back on fuel storage tank 

14. Puts lid back on fuel tank 

15. Unscrews oil tank lid 

16. Wipes short dipstick clean 

17. Inserts short dipstick 

18. Does not wipe dipstick 



19. Adds oil from oil storage tank 

20. with oil funnel 

21. Cleans short dipstick 

22. Inserts short dipstick again 

23. Does not wipe dipstick 

24. Puts lid back on oil storage tank 

25. Puts lid back on oil tank 

26. Checks to see if valve is in bypass position spot 
(by looking in bucket) 

27. Puts plug in battery socket 

28. Turns on ignition switch 

29. Turns bypass valve on 

(Prompt him to shut it off if he doesn't do it immediately) 

30. Turn valve to bypass 

31. Turn off ignition 

32. Does not pull plug 

33. Cleans up funnels and spills of oil and fuel 



Total Points 



LESSON III 



Assembly 
Scoring Checklist 



Name Date 

1. Takes packing nut and stem 

2. Screws packing nut to top of stem 

3. Takes gate 

4. Screws gate on bottom of stem 

5. Winds gate all the way up stem 
( Prompt if Step 5 performed incorrectly) 

6. Inserts gate and otem assembly into body of valve 

7. Screws ate and stem assembly on to body of valve 

8. Inserts handle onto top of stem 

9. Screws handle onto top of stem with handle nut 

10. Screws on first 3/4" nipple 

11. Screws on second 3/4" nipple 

12. Checks assembled valve to see if parts are fitted tightly 
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LESSON II 



Name of Trainee 



Date 



Scoring Checklist for Making a Full Face Gasket 



1. Uses round end 

2. of ball peen hammer 

3. and correct size gasket material 

4. and taps out one bolt hole 

5. lightly. 

6. Inserts bolt in hole. 

7. Taps out diagonally opposite bolt hole and inserts bolt. 

8. Taps out remaining bolt holes in any order 

9. lightly. 

10. Taps out inside circumference of flange 

11. using round end of ball peen hammer 

12. lightly. 

13. Taps at outside circumference of flange 

14. using flat end of ball peen hammer 

15. lightly. 

16'. Remove tapped out pieces of gasket material from gasket 
and flange. 

17. No frayed edges in final product. 

18. No gouges or scratches in final product. 

19. Does not ruin or throw away any pieces of gasket material. 
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R-K Research & Systems Design 
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